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ABSTRACT 

Artificial neural systems (ANS), also known as neural networks, are an attempt to develop computer 
systems that emulate the neural reasoning behavior of biological neural systems (c.g. the human brain). As such, 
they are loosely based on biological neural networks. The ANS consists of a series of nodes (neurons) and 
weighted connections (axons) that, when presented with a specific input pattern, can associate specific output 
patterns. It is essentially a highly complex, non-linear, mathematical relationship or transform. These constructs 
have two significant properties that have proven useful to the authors in signal processing and process modeling, 
noise tolerance and complex pattern recognition. Specifically, the authors have developed a new network learning 
algorithm that has resulted in the successful application of ANS's to high speed signal processing and to 
developing models of highly complex processes. Two of the applications, the Weld Bead Geometry Control 
System, and Welding Penetration Monitoring System is discussed in the body of this paper. 

INTRODUCTION: ARTIFICIAL NEURAL SYSTEMS 

Artificial Neural Systems (ANS) are loosely based on biological neural networks offer a computer 
technology that is a useful tool in process modeling and signal processing. The ANS consists of a series of nodes 
(neurons) and weighted connections (axons). As with a biological neural network, the assignment of the values of 
the weights and the size and configuration of the network is the key to a successful net. Unfortunately, we have 
only begun to understand the inner workings of these constructs. Consequently, relatively crude tools are 
currently employed to develop working networks. 

Typically, the approach to model development is to develop a thorough understanding of the underlying 
basic scientific or engineering principles of the process. Then, a model is developed that is based on 
mathematical relationships inherent in the process parameters. The principal drawback with this approach is the 
time and effort required to develop an understanding of these basic scientific or engineering relationships. 
Depending on the complexity of the problem, it can take many years and an extensive research program to 
develop the relationships. Often, instead, a number of simplifying assumptions are made and an approximate 
model is developed. That approach, while being an expedient method for developing an approximate model that 
could be useful, provides only a theoretical approximation that may not be valid for the actual problem 
application. 

The artificial neural system, when a network can be found to solve the problem, provides an accurate 
model of the process or signal. Neural networks are empirical bases systems that, when presented with a specific 
input pattern, can associate specific output patterns. It is, essentially, a highly complex, non-linear, mathematical 
relationship or transform. However; it is not necessary for the developer of such a system to understand the basic 
underlying principles of a process in order to develop a highly accurate ANS based model of the process. Thus, in 
this way it is quite different from other mathematical modeling approaches. 

Basic Principles 
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CONCLUSIONS 


A fuzzy-neural network was successfully implemented for automatic camera platform positioning, 
eliminating the need to manually adjust FLC parameters for optimum performance. The simulations 
conducted in this study demonstrated that when structured knowledge and learning ability are combined, 
the result is robust performance that improves over time. The fusion of these two types of machine 
intelligence represents an important step in the evolution of intelligent control systems. 

The FNN may be applied to a wide range of non-linear control systems, particularly where fuzzy 
control has already proven successful. In any system whose desired behavior can be described through a 
series of fuzzy rules, those rules can be translated into a network and thereby achieve the ability to adapt. 
The learning algorithm does not add an excessive amount of computation to an existing FLC, and thus it 
should be feasible for real-time applications where the system can learn "on the fly". Such experiments 
are planned for this system in upcoming research. 
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T , he . pr °^ le ' n of ANS s 18 to dec,de how ma ny nodes and connections are needed to model a specific 
problem to decide how to configure them, and to decide the specific values of the connection weights and the 

° nS i* 31 CXISt W,thm Ae network - Flgure 1 shows a schematic diagram of a neural network and 
g 2 is a simple representation of the weights, transfer functions, and the mechanisms of network operation 
Ttere is no direct known correspondence between the network parameters and it’s operation and the problem to be 
modeled by the network. As a consequence, there is currently a lack of good mechanisms which can be used to 
assign the weights and transfer functions in the network so that it can solve a problem. The methodology of 
finding a proper net configuration and weights to model a given problem is called the "Learning Algorithm" 
There are many learning algorithms that have been developed. They all have some advantages and restrictions In 
the modeling area, the back-propagation method is the most popular one. 

u The r baCk Pr ° pagat ‘ 0n method assumes that the m weight space for an optimum, or near optimum 
network configuration can be accomplished as an iterative search using the error gradient, (i.e. slope of the error 
surface) m L 2 space. That is, a senes of moves are accomplished on the multi-dimensional error surface using 
approximately the maximum mean squared error gradient direction as the move direction at each iteration. The 
error, so c e the delta in the network is defined as the mean squared error between the desired output 
representation and the actual output given the current weight matrix values. By calculating the maximum gradient 
. ® , C ° r any glVen tra * n ' n 8 example (set of input and corresponding output patterns), the weights are 

„ S ° hat net m ° ves along that gradient direction in each presentation of the training example to the 

network. Using this procedure, the network slowly "learns" to associate all of the training example input patterns 
with the correct corresponding output patterns by finding a global minimum on the error surface Since the 
primary ving force in back-propagation method is the mean squared error - delta, it also called the "delta rule." 

„ r ™' S basic ' back Propagation learning process has several significant drawbacks. First, the optimum 
o" flg u n (i e. number and relative location of hidden representation units or nodes) cannot be pre-determined 
and yet, needs to be pre-assigned by using an "educated guess" in order to use this procedure. Since the node 
configuration can significantly affect the operation of the network, this will at best lead to a long series of re-trvs 
and, at worse, to no useful network at all. Second, this process is very slow and the rate of learning (convergence 
to near zero error) is set arbitrarily - traditionally at a value between zero and one. Even though several 
researches are going on, no currently known method for predetermining the learning rate (gain term) will 
consistently choose an optimum value, and the optimum value is significantly influenced by the specific problem 
emg presented to the network. Third, it has been shown, that it generally, requires a larger network to "learn" a 
problem than is required to solve the problem. Based on many study, people generally know that the net could be 
mm to smaller size. However, there is no known method of reducing the size of the network optimally after 
training to optimize the net performance. Finally, learning instabilities exist in nearly every problem which will 
cause the network to stop learning (converging). One of these instability types is known as a local minimum A 
local minimum ,s a depression in the error surface, but one which is not the best minimum or lowest error 
position. In its search routine, the network algorithm may fall into a local minimum and since the error gradient is 
toward the local minimum from all directions, may not be able to exit from it. 

The Delta Activity Network 


A new method was developed by the authors for training neural networks that has been shown to 
overcome all of the known problems with the back-propagation method, while maintaining the inherent stability 
and known network development capabilities of that method. This network was developed through the use of a 
thermodynamic model of the network operation which included both the delta energy and the activity or kinetics 
ot the network. The technique, known as the Delta-Activity Network (D-A Net), has been used on several 
applications ranging from high speed signal processing to vision systems. 
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The authors have been studied the learning behaviors of artificial neural networks for years. Based on the 
thermodynamic and kinetic model of the learning, we discovered that the learning is not only driven by the delta, 
but also by another important factor - the activity. The activity is a kinetic measurement about how good foe 
neural networks is willing to learn at certain stage of the learning. Sometimes, even the de ta is l«ge, b* f _ foe 
system has low activity, it won't learn. This is the case that local minimum or other kinds of learning instabilities 
accrues Thus the ddta-activity algorithm will watch the activity closely during the learning, maintain a 
reasonable activity while push the learning rate as high as possible. 

The D-A Net has achieved learning rates as high as 1000 times that of the back propagation method while 
also preventing the network from falling into learning instabilities. Research conducted on thls ^^ UC ^ 
confirmed the existence of at least three types of learning instabilities (local minimum being one of them) and the 
algorithm can avoid all three instabilities. In addition, using activity as the critical, the system configures itself 
dynamically during the learning process and so it can produce a near optimum network size for operation, often 
much smaller than the network needed to "leam" the problem. 


This network has been successfully applied to model many real manufacture application problems. These 
applications are: 

• ANS models of weld bead geometry in several different welding processes and joint types 

• ANS model to predict weldment mechanical properties for SAW welding 

• ANS model of weld metal hot cracking 

• ANS model of heat flow for arc weld 

• ANS based weld seam tracking system 

• ANS based welding penetration monitoring system 

• ANS based acoustic emission signal analysis to detect the welding failures 

• ANS based underwater acoustical signal detection and classification 

• ANS based ultrasonic signal process to detect helicopter tail rotor gearbox fault 


In this paper, we will select few examples from above and given more detail discussions. 


An ANS Model of Weld Bead Geometry 


Background 

The mechanical properties, geometry, and appearance of the weld bead are the three major characteristics 
of the final quality of a weld. They are interactive with each other. The weld bead geometry directly effects the 
mechanical properties of the weldment. A good weld bead geometry is necessary to insure a good quality weld. 
The weld bead geometry can be effected by various welding variables. These variables are. 

• The process variable, such as the type of welding process, the welding current, voltage, travel speed and 
wire feed speed. 

• The material variables, such as the type and size of base metal and till metal. 

• The joint configuration, such as the type and geometry of welding joint, and the welding position. 

To model the bead geometry, an ANS has to perform the transformation from these variables to the bead 
geometry. 
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Approach 


«/ a- F ^ Uf DeUral networks have developed to model Gas Tungsten Arc Welding (GTAW), Gas Metal Arc 
Welding (GMAW), Flux Cored Arc Welding (FCAW), and Submerged Arc Welding (SAW). To simplify the 
problems, some variables are fixed. The GTAW net is developed based on a stainless steel butt joint weld The 

nTZT “* CUITent ’ V ° ltagC * Wire feed Speed ’ travel s P eed * P ,ate thickness, and joint gap. The FCAW and 
OMAW nets are based on a plain carbon steel fillet weld. 


Several experiments were done to generate the training and test sam ple 
number of training samples used by each net: 


The following table lists the 


Table I 

The Number Of Training Samples Used By Each Net. 

Mst Number Of Training Samples 

GTAW 28 

FCAW 34 

GMAW 14 

SAW 38 

Each sample was sectioned and the cross section of the weld bead was measured following the definition 
to generate the training and test examples. 

We have to point out here how few samples the ANS needed to develop a complete model of the weld 
bead geometry. For instance, in the GTAW case, if we used the Taguchi method, which is known as the best 
statistical method, we needed at least 150 samples to develop a complete model. Using ANS approach only 28 
samples were needed. ’ 

Another important function of the shell is its ability to give the user a suggestion about further 

experimental points. This function can insure the generation of a complete model with minimum amount of 
samples. 

Results and Discussions 


Figure 1 is the configuration of the GTAW net. The net takes 6 inputs and generates 4 outputs It finally 
configured itself to have three hidden layers with 7 nodes in each layer. 

Figure 2 shows the interface shell of GTAW net. There are two rows of mouse sensitive slide bars on the 
screen that can be set by the user or observed by the user. These bars can be moved to new values by using the 
mouse to slide" the bar. The first row of the slide bars are the outputs of the net, which are the bead width, 
height, penetration, and the bottom bead width. The second row of slide bars are the input variables they are the 
current, voltage, travel speed, wire feed speed, gap, and the plate thickness. At the top left comer is a graphical 
simulation of the bead cross section. When the net is running in the forward model, the user can slide the input 
bars, the output bars will change correspondingly and the graphical simulation will animate the bead cross section 
simultaneously. When the net is running in the inverse model, the user can set the bead geometry by sliding the 
top row bars and the gap and thickness by sliding the right two bars on the bottom row. The net will find the 
closest match bead and give the welding parameters which can be read from the bottom row of bars. The whole 
system runs in real time on a personal computer. Thus, this represents a very powerful planning tool for a welding 
engineering workstation as well as a complete model for real-time intelligent control. 
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Figure 2 and 3 show the performance of the GTAW model. In these two figures, the test sample, which 
the net has not been trained with, is compared with the prediction of the net. A very close match between the 
predictions of the net model and the actual welds is shown. 

Figure 4 shows the GTAW net running in the inverse model. There are two graphical representations of 
bead cross section on screen. The user specified bead geometry is displayed as the top one. The bottom one is the 
closes match that the net found. 

The GMAW network has similar interface as the GTAW system. However, it also takes two discrete 
inputs: type of electrode and type of shielding gas. And it also generate two additional discrete outputs: the spade 
and the silicon slag appearance. FCAW and SAW nets also have a similar interface. 


These model networks are the heart of an intelligent control system for welding applications. The 
objective of an intelligent control system is not to control the primary independent welding parameters (e.g. 
voltage, current, travel speed, etc.) but to control the final weld quality. That is, control of weld quality 
parameters such as bead appearance, penetration, amount of spatter, etc. is the goal of an intelligent control 
system. Sensor data is not particularly useful unless it can be both analyzed and used to control the weld quality 
parameters. These model networks provide that capability. 

The results of these models are being used to develop fuzzy logic based control systems on two projects 
including development of an intelligent automated welding system for the United States Navy Manufacturing 
Technology Program called WELDEXCELL. 


An ANS Based Welding Penetration Monitoring System 


Background 

The development of real-time sensing and control techniques for weld penetration has been an active area of 
research for a number of years. A wide range of techniques have been employed, including pool oscillation 
measurements, surface infra-red measurements, determinations of the light emitted on the back side of a weld, 
spectroscopic determination of the presence of tracer elements, and the use of vision techniques to measure weld 
pool geometry, with the objective of relating to weld penetration. 

The success of each of these methods has been limited, and all require the use of specialized transducers. The 
most successful technology to date appears to be the use of photo detectors to measure the amount of light 
produced at the back side of the weld, which is then correlated with penetration. Unfortunately, back side access is 
required, limiting suitability for many applications. Back side monitoring is difficult to make and it is costly. 

A pproach 

Recently, several research works (see ref. [1-3]) have been done to study the weld pool oscillation. The researches 
show that a full-penetrated weld pool can be modeled as "rubber band" (no rigid base to support it) while a 
partially penetrated weld pool can be modeled as a half sphere. As a consequence, the pool oscillation pattern will 
be different in two models. Several methods have been applied to sensor the pool oscillation pattern in real-time, 
such as the lasers, and trying to relate them with the weld penetration. Unfortunately, the sensors are usually too 
expensive, and the data from those sensors are too "noisy" to process. These barriers limit the success of those 
approach. 
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e authors developed a new approach to detect the pool oscillation pattern and related it to the penetration in 
real-time. This uses a very m-expensive sensor, the voltage sensor, and the Delta-Activity ANS to process the 
sensor data. The theory behind this is that when the weld pool oscillates, the voltage between the base metal and 
welding torch tip will changed accordingly. By monitoring the voltage data in a continues time sequence, the pool 
oscillation pattern could be detect. However, the voltage data obtained are usually very noisy and contained much 
more information than just the pool oscillation parterre. Delta- Activity ANS is played a important role here to 
successfully filter out the pool oscillation pattern information from the rest . 

The voltage signal is collected at a 1000 Hz sampling rate. A neural network was developed off-line. This neural 
network takes 100 data points as the inputs. The output of the ANS will indicate that either a full penetration (0) 
or a lack of penetration (1) is been detected. The experimental work used GTAW process for carbon and stainless 
steel butt joint welding on various plate thickness. Total 25 welding samples were used to train the net. 

Result 

After training finished, the ANS is installed on a 486 based PC. A GUI is developed to run the penetration 
monitoring system. If a lack of penetration is detected, the system will turn on a warning light and/or shutdown 
t e welding process. Eventually, combining with the weld bead geometry model, the control loop could be closed 
and a corresponding welding parameter could be adjust automatically to insure a full penetration welding This 

system is installed in AWI's workshop. Figure 5 is the computer GUI and Figure 6 shows that the system is 
monitoring a welding. 
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Figure 1. The Configuration of neural network for GTAW modeling 
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Figure 2. Neural network solution for the GTAW butt weld. Selected parameters of: 

voltage = 8.8 volts, current = 150.0 amperes, travel speed = 5.0 inches per min ute, 
and wire feed rate = 55 inches per minute. In addition, the weld is being made in 
material of thickness = 0.125 inches and with a gap of 0.035 inches. The resultant 
shape parameters and bead graphic are shown in the interface. 
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Figure 3. The actual weld bead cross section which was welded using the welding 
parameters described in the Figure 2. 
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Figure 4. The GTAW network running in inverse model 
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Figure 5. The GUI of real-time welding penetration monitoring system 



Figure 6. The GTAW Welding Penetration Monitoring System 
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Neural networks are interdisciplinary studies about animal brains. These 
have improved AI towards the 6th Gen Computers. Enormous amounts of 
resources were poured into this R/D awaiting for breakthroughs. International 
Neural Network Society held two Conferences attended by thousands each year, 
pushing the ultimate & exciting frontier of computing & info-tech. — our bio-brains. 
Wavelet Transforms (WT) replaced Fourier Transforms (FT) favorably in 
every known Wideband Transient (WT) cases that began with the discovery of WT 
in 1985— the French geological exploration for oils by means of seismic wave 
imaging. The list of successful applications has the earth quake prediction, the Radar 
ID, speech recognitions, stock market forecasts, FBI finger print image compressions, 
telecommunication ISDN-data compression. More, the billion dollar medical- 
industrial has applications that still await the perfection— the intelligent heart beat 
pace-maker, the echoless hearing aids, in vivo constant level drug-dispensor, etc. 


1. Introduction 

A surging interest in neural nets began in 1980, when J. Hopfield wrote 
articles in Proc. Nat. Acad. Sci. p.2554 (1992) & p.3088 (1994), although pioneers such 
as Grossberg, Amari, Widrow, Kohonen, Fukushima, Anderson, Freeman, von der 
Malsburg, Rumelhart, Werbos, Carpenter, Cooper, have made significant 
contributions earlier. His model is simple for engineering because of interacting 
magnets (i.e. neuron points to the north for yes-vote, the south for no-vote) having 
simple matrix interconnects. Neurons are of McCulloch-Pitts (M-P) threshold logic. 

2. Mathematical Foundation of Artificial Neural Networks 

M-P Model of the ith neuron (in alphabetic order: u-input & v-output): 


v i= 0(^1 = 1/(1+ exp(— u i )); Grossberg' s: (d/dt)v i = c (vp ofu^) (1) 

where each voting v- is weighted by previous memory W^ as the net input up 
Uj= £j Wjj Vj + 0; Hopfield's: (d/dt)u-= b (u-- Zj W^j Vj - 0 ); (2) 

Wjj= VjVj; Hebb's: (d/dt) Wjj= a (W^-v^Vj) (3) 
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